A convolution neural network was trained via transfer learning to identify Bactrocera species using wing images. Additional tephritids were included but only identified to genus. Threefold cross-validation was used to achieve an accuracy of 97% (845/870) at predicting class. The image set comprised 870 images in 18 classes. The image set was sorted into three folds using stratified round-robin cross-validation. Each set had a 2/3 training, 1/3 test split.
Images were prepared by:
Transfer learning was used to retrain an Xception model initially trained on the Imagenet dataset. The model was retrained three times, each run was initially set for 200 epochs but with an early-stopping callback and learning rate of 1e-04. Each model was fine-tuned at a learning rate of 1e-05, again with early-stopping set.
Images were augmented to prevent over-fitting and increase the robustness of the model to variation in additional images added later. Image augmentations included:
The model achieved an average accuracy of 97% (845 correct/25 wrong) across the three runs. Each model stopped before reaching 200 epochs with accuracy and loss both plateauing (Figure 1).
Fig. 1: Accuracy and loss of the three cross-validated Xception models.
Validation accuracy varied between classes, which was to be expected given the relatively small dataset for some classes.
| class | precision | recall | f1-score | support |
|---|---|---|---|---|
| austrotephritis | 0.96 | 1.00 | 0.98 | 48 |
| bactrocera_curcubitae | 1.00 | 1.00 | 1.00 | 19 |
| bactrocera_distincta | 1.00 | 1.00 | 1.00 | 36 |
| bactrocera_dorsalis | 0.90 | 0.97 | 0.93 | 29 |
| bactrocera_facialis | 0.86 | 0.69 | 0.77 | 26 |
| bactrocera_frauenfeldi | 0.95 | 1.00 | 0.98 | 20 |
| bactrocera_kirki | 0.94 | 0.88 | 0.91 | 17 |
| bactrocera_melanotus | 0.96 | 0.92 | 0.94 | 24 |
| bactrocera_passiflorae | 0.75 | 0.78 | 0.77 | 23 |
| bactrocera_psidii | 1.00 | 0.50 | 0.67 | 4 |
| bactrocera_tryoni | 0.79 | 0.88 | 0.83 | 17 |
| bactrocera_umbrosus | 1.00 | 1.00 | 1.00 | 19 |
| bactrocera_xanthodes | 0.87 | 0.96 | 0.92 | 28 |
| ceratitis | 1.00 | 1.00 | 1.00 | 32 |
| procecidochares | 1.00 | 1.00 | 1.00 | 19 |
| sphenella | 1.00 | 1.00 | 1.00 | 10 |
| trupanea | 1.00 | 1.00 | 1.00 | 471 |
| urophora | 1.00 | 1.00 | 1.00 | 28 |
| accuracy | NA | NA | 0.97 | 870 |
| macro avg | 0.94 | 0.92 | 0.93 | 870 |
| weighted avg | 0.97 | 0.97 | 0.97 | 870 |
The 25 images in the table below were incorrectly identified during validation.
| file | actual | pred | score | test_img | prediction_exemplar_img |
|---|---|---|---|---|---|
| bactrocera_dorsalis_Dacus dorsalis_wing.png | bactrocera_dorsalis | bactrocera_tryoni | 0.5121685 |
|
|
| bactrocera_facialis04254492.png | bactrocera_facialis | bactrocera_kirki | 0.7213987 |
|
|
| bactrocera_facialis_Bactrocera facialis_wing.png | bactrocera_facialis | bactrocera_passiflorae | 0.9962202 |
|
|
| bactrocera_melanotus_Dacus melanotus_wing.png | bactrocera_melanotus | bactrocera_facialis | 0.3934056 |
|
|
| bactrocera_passiflorae04204131.png | bactrocera_passiflorae | bactrocera_tryoni | 0.8161860 |
|
|
| bactrocera_passiflorae04254454.png | bactrocera_passiflorae | bactrocera_tryoni | 0.5324097 |
|
|
| bactrocera_psidii_Bactrocera psidii_wing.png | bactrocera_psidii | bactrocera_frauenfeldi | 0.5140736 |
|
|
| trupanea04254299.png | trupanea | austrotephritis | 0.9999244 |
|
|
| trupanea04254411.png | trupanea | austrotephritis | 0.9998648 |
|
|
| bactrocera_facialis04254491.png | bactrocera_facialis | bactrocera_passiflorae | 0.8368283 |
|
|
| bactrocera_facialis04254494.png | bactrocera_facialis | bactrocera_dorsalis | 0.5409914 |
|
|
| bactrocera_facialis_Bactrocera facialis_wing.png | bactrocera_facialis | bactrocera_dorsalis | 0.5713534 |
|
|
| bactrocera_kirki04257143.png | bactrocera_kirki | bactrocera_xanthodes | 0.5003277 |
|
|
| bactrocera_melanotus04254483.png | bactrocera_melanotus | bactrocera_xanthodes | 0.8524919 |
|
|
| bactrocera_passiflorae_Bactrocera passiflorae_wing.png | bactrocera_passiflorae | bactrocera_tryoni | 0.9416133 |
|
|
| bactrocera_psidii_Bactrocera psidii_wing.png | bactrocera_psidii | bactrocera_xanthodes | 0.4697781 |
|
|
| bactrocera_tryoni04204129.png | bactrocera_tryoni | bactrocera_passiflorae | 0.5359533 |
|
|
| bactrocera_tryoni04248825.png | bactrocera_tryoni | bactrocera_dorsalis | 0.5752694 |
|
|
| bactrocera_facialis_Bactrocera facialis_wing.png | bactrocera_facialis | bactrocera_passiflorae | 0.4918163 |
|
|
| bactrocera_facialis_Bactrocera facialis_wing.png | bactrocera_facialis | bactrocera_passiflorae | 0.5807632 |
|
|
| bactrocera_facialis04254498.png | bactrocera_facialis | bactrocera_xanthodes | 0.6963255 |
|
|
| bactrocera_kirki04254472.png | bactrocera_kirki | bactrocera_melanotus | 0.3627877 |
|
|
| bactrocera_passiflorae04204141.png | bactrocera_passiflorae | bactrocera_facialis | 0.5044423 |
|
|
| bactrocera_passiflorae_Bactrocera passiflorae_wing.png | bactrocera_passiflorae | bactrocera_facialis | 0.5432149 |
|
|
| bactrocera_xanthodes04254470.png | bactrocera_xanthodes | bactrocera_passiflorae | 0.5603202 |
|
|
Value for each species is the training accuracy. I am uncertain how to interpret the activation maps. While the wing itself appears to be the most salient object in each image, attention to the finer details within each wing may be needed for better model performance. Is this a result of the small dataset?